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(57) A method and apparatus extracts a foreground 
image and a background image from a color document 
image. The color document image is divided into a plu- 
rality of multiscaled grids. Each of the grids includes a 
plurality of blocks and the resolution of the plurality of 
blocks increases for each successive grid. The back- 
ground color and the foreground color of each block of 
the largest grid is first determined. Then the background 



color and the foreground color of each block of the small- 
er grids are determined iteratively, using biasing infor- 
mation from each block's corresponding block, until the 
background and foreground colors of all of the blocks of 
all of the grids have been determined. Finally, the fore- 
ground and background images are formed from the de- 
termined background color and foreground color of each 
block of the smallest grid. 
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SUMMARY OF THE INVFMTinM 



BRIEF DESCRIPTION ) Q F THE DRAWINGS 

Rq i S - «o CK ? a rt 9 T ° f , 3 C ° mpUter that im P |ements one embodiment of the present invention 
document image^ * 3 C ' US,erin9 PrOCSSS M b *^™« ''-ground colors from a oico.or 

Fig. 3 illustrates a typical color document image. 
Fig. 4 illustrates a color document divided into rectangular blocks 
Fig. 5 illustrates part of a second grid of a color document 
Fig. 6 illustrates part of a third grid of a color document 

and^Zdtr^^^^^ 

and ZlZZl^ir^Z CZ? IT" * m^ ° < "™ n, <* - •*""» '° «*■« 



DETAILED DESCRIPTION 



invention The compute, 10 inc udes. £S i?IT,?ilr™ I t, " nplomenl * "» «*odi™nt ol th. preen! 
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CPU from MIPS Corporation. 

Computer 10 further includes a plurality of storage devices The storage devices include a disk drive 16, read-only 
memory ("ROM") 20 and random access memory ("RAM") 18. Coupled to computer 10 is a monitor 26 for displaying 
output and a keyboard 22 for providing input to computer 10. Further coupled to computer 10 is a scanner 24 that 
s scans a color document and converts it into a digitized color document image. The color document image is stored in 
a storage device of computer 10. The color document image that is stored in computer 10 comprises a plurality of 
image pixels. 

In one embodiment of the present invention, the functionality of steps that will be described herein are implemented 
as software instructions that are stored on a storage device of computer 10 and executed by CPU 14. In another 
io embodiment, the steps are implemented using hardware devices, or a combination of software and hardware devices. 
The document images used in the steps are formed by scanning a document using scanner 24. 

In order to describe how to extract a foreground and background image from a varied, multicolor image, it is useful 
to first describe how to extract the background and foreground color from a bicolor document image (i.e., an image 
with a quasi-uniform background and foreground color). 
is One approach to extract the foreground and background color is to create a three-dimensional ("3-D") color his- 

togram of the bicolor image. A 3-D color histogram, which is well known, includes for each possible color in the image 
a count of the number of pixels in the image having that color. These counts are organized along three axes: red, green 
and blue, each ranging in intensity from 0 to 255. Therefore, at the origin (0, 0, 0) each of the three intensities are 0 
and the corresponding color is black. Likewise, at the far corner of the 3-D color histogram is the point (255, 255, 255) 
20 corresponding to the color white. 

A 3-D color histogram of a bicolor document will have two peaks, corresponding to the background and foreground 
colors. From these peaks, the background and foreground colors can be extracted. Typically, the highest peak is the 
background color and the lowest peak is the foreground color because there is usually more background than fore- 
ground in a document. 

25 In the alternative, extraction of the background and foreground colors of a bicolor document can be achieved by 

implementing a clustering process that can be executed on computer 10 of Fig. 1 . Fig. 2 is a flowchart of a clustering 
process that extracts the background and foreground colors from a bicolor document image. The clustering process 
of Fig. 2 (the "bicolor process") is derived from the "K-means" algorithm disclosed in J. MacQueen, "Some Methods 
for Classification and Analysis of Multivariate Observations", Proceedings of the Fifth Berkeley Symposium on Math- 

30 ematics, Statistics and Probabilities, Vol. 1 , pp. 281-297 (1 967). 

At step 40 of the bicolor process, the background and foreground colors, which are stored in computer 10 as 
variables, are initialized to white and black, respectively. This insures that the extracted color with the highest luminosity 
will be returned as the background color, and the extracted color with the lowest luminosity will be returned as the 
foreground color. 

35 At step 42, for each pixel in the image, a determination is made as to whether that pixel is a foreground or back- 

ground pixel by comparing the distances between the pixel color and the current foreground and background colors. 
In one embodiment, the distances between colors are determined by calculating the square root of the sum of the 
squares of the differences between the red, green and blue coordinates of the colors. If the pixel color is closer to the 
current foreground color, the pixel is determined to be a foreground pixel, and vice versa. 

40 At step 44 the current foreground color is updated by computing the average color of all the pixels that were 

determined to be foreground pixels in step 42. Likewise, the current background color is updated by computing the 
average color of all the pixels that were determined to be background pixels in step 42. 

At step 46 it is determined whether the process has converged. Step 42 must be executed at least twice before 
the process is determined to have converged. The process is determined to have converged if the pixels determined 

45 to be foreground pixels at step 42 are the identical pixels that were determined to be foreground pixels when step 42 
was previously executed (consequently, the pixels determined to be background pixels at step 42 must be the identical 
pixels that were determined to be background pixels when step 42 was previously executed). If the process is deter- 
mined to have converged at step 46, then the current background and foreground colors are considered the extracted 
background and foreground colors of the bicolor document. If the process has not converged, step 42 is repeated. 

so The bicolor process of Fig. 2 typically converges very quickly. The bicolor process can be implemented to converge 

even faster with the use of stochastic approximations that are disclosed in L. Bottou et at., "Convergence Properties 
of the K-Means Algorithms", Advances in Neural Information Processing Systems, vol. 7, MIT Press (1995). 

Fig. 3 illustrates a typical color document image. The color image 50 includes text 58 and various color pictures 
52, 54 and 56. Color image 50 also includes a white background 60 and a colored background 62. Because a typical 

55 color document image such as color image 50 includes more than two colors, the bicolor process of Fig. 2, by itself, 
will not properly extract the background and foreground images of document 50. 

One improvement on the bicolor process is to divide the color document image into rectangular blocks of pixels 
of a single size. Fig. 4 illustrates color document image 50 divided into rectangular blocks 70, 71 , 72, etc. The bicolor 
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bl°com h l F H 9 to 2 , Can '"I" b t PSrf T ed f ° r 6aCh b ' OCk and thS back 9 rou " d foreground colors of each block can 

j^^^'.s and ,ore9round imase However ' this method is M by «— 

Specifically, the blocks should be small enough to capture a foreground color change, for example of a red word 

IT er rr, blaCk t6Xt - ThS SiZS ° f ,he Sma,lest Characters should ^ore be the maximize of the bl^ks 

However, rf the blocks are too small, the number of blocks entirely located outside the text area, fT^SmpteSSS??" 

ZZTluS^; C T in °?' y , baCk9r ° Und PiX6,S - BIOCkS may alS ° be M * 'ocated insideTetk Sab g 
li. ™ t . h * , nta ' n ° nly fore 9 round P jxels Tn e bicolor process fails in both cases to determine a pair of 

nriH The ,P reSe " t Mention overcomes the above problems by dividing the document 50 into a plurality of multiscaled 
grids, with each grid including a plurality of blocks. The blocks of each grid are the same size ?he b tock Z of eaS 
successive gnd » a fraction of the size of the b.ocks of the previous grid (i.e., each successive grid has anTnclstnS 

Th^ir^ P ' e ' 4 j" UStra,es Color ^ument image 50 divided into a first grid that includes the largest size blocks 
Th.s first gnd ,s referred to as the "largest" grid. The second grid is formed by dividing each block of ?he irs^qrid into 

nSLVT? t ! 9 ' 5 " IUStra,eS Part ° f 3 SeC ° nd 9rW f ° r dOCUment ima 9 e 50 ' n Fi 9- * block 70 from F g 4 s 
further divided into a plurality of smaller blocks 81 , 82, 83, etc. Each block in Fig. 4 is similarly divided into smaTer btocks 

int. f W '^' ? " ? Strat6S Parl °' 3 thlrd 9hd fordoc ument 50. in Fig. 6, block 81 from Fig 5 is "rth dMded 

into a plurality of smal.er blocks 91 , 92, 93, etc. Each block in Fig. 5 is similarly drvided into smal.er blocks Therefore 
document .mage 50 » u Itimate.y divided into a plurality of blocks the size of blocks 91 -93 when it is divided by the S 

X~ ^TtLT: f " fU '"f' int ° SUCC6SSiVe 9fidS Unti ' the b,0Ck Size of ,he -allestt id is a Ted^ 
* document imlge ""^ Sh ° U ' d be si >^ s ™»" »an »• -» of the smallest readable character in 

a , . FiQ 7 iS 3 « owchart of the steps performed by one embodiment of the present invention to extract the background 
and foreground image of a color document image using muttisca.ed grids. A, step 100, the document image is Sed 

ZXl^sTZT, 9 T r ^Tk R9S - 4 " 6 ' Wi,h 6aCh SUCC6SSiVe 9rid havin 9 a " increasing ^esoluton of 
blocks. The number of grids .s determined by first determining the desired size of the smallest blocks 

At step 102 the background and foreground colors of each block of the largest grid are determined The lamest 
gnd is the gnd that includes the largest size blocks (i.e., the grid wtth the lowest resolution) Any method can be used 
^T^"^ baCk9r ° Und ^ ^ 3 "° hiSt ° 9ram method and « he ^ P-e- pTe 

At step 104, the background color and the foreground color of each block in the next smaller arid is determined 
The determnat.cn ,s made for a current block by using biasing information from the correspond^ 

can b'e'thl f 6 T 9fid COntainin9 thS Cen,9r ° f ,he CUrrent bl0ck > The bias ^ 

can be the foreground and background colors determined for the corresponding block of the previous grid ThereTo e' 
the determination of the background and foreground colors of block 81 of Fig. 5 can be biased with the bLkqround 
and foreground colors of block 70 that were previously determined oacKground 
„h- The t co " es P° ndin 9 block * 'arger than the current block and therefore covers several blocks of the current arid 

are c^Vt^J^T 31 ? 1 'hT^ ^ l ° 6nSUre that ,he foreQround and background colors of the current block 

b iks P colred L Z blOCk PrOV ' d ! S S,r ° n9 6VidenCe that the current block -9nmcantly differs from the adjacent 
blocks covered by the same corresponding block in the previous grid 

In one embodiment, the colors are biased in step 104 using the following equation: 

50 New color of current block = 

a(calculated color of current block) 
+ (1-a)(color of corresponding block 

55 

of previous grid), 

where 0 < a < 1 . 
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At step 106, it is determined whether there are any smaller grids than the current grid. If there are, step 104 is 
repeated. If there are no smaller grids, at step 108 the background and foreground color images are formed by com- 
bining the determined foreground and background colors of the blocks in the smallest grid. The smallest grid is the 
grid that includes the smallest size blocks (i.e., the grid with the highest resolution). 

Fig. 8 is a flowchart of the steps performed in another embodiment of the present invention to extract background 
and foreground images from a color document image. The embodiment of Fig. 8 implements the bicolor process of 
Fig. 2 and a modification of the bicolor process to determine background and foreground colors of each block. 

At step 1 20, the document is divided into a plurality of multiscaled grids as shown in Figs. 4-6, with each successive 
grid having an increasing resolution of blocks. The number of grids is determined by first determining the desired size 
of the smallest blocks. 

At step 1 22, the foreground and background colors of each block in the largest grid is determined using the bicolor 
process of Fig. 2. 

At step 124, the determination of the background colors of the next smaller grid is initiated. This determination is 
performed using a modification of the bicolor process of Fig. 2 in steps 126-1 32. 

At step 126, the foreground and background colors of each block in the grid are initialized to the foreground and 
background colors that were determined for the corresponding block of the previous grid. For example, block 92 of 
Fig. 5 is initialized with the background and foreground colors that were previously determined for block 81 . 

At step 128, for each pixel of each block, a determination is made as to whether that pixel is a foreground or 
background pixel by comparing the distances between the pixel color and the current foreground and background color 
of that block. 

At step 1 30, the foreground color of each block is updated using a weighted average according to the following 
equation: 

New foreground color of block = 
a(average color of all foreground 
pixels of block) + (1-a)(foreground 
color of corresponding block); 

and 

New background color of block - 
a (average color of all background 
pixels of block) + (1-ct)(background 
color of corresponding block), 

where 0 < a < 1 . 

At step 132, for each block in the current grid it is determined whether the process of Fig. 8 has converged. Step 
128 must be executed at least twice before the process is determined to have converged. The process is determined 
to have converged if the pixels determined to be foreground pixels at step 128 are the identical pixels that were deter- 
mined to be foreground pixels when step 128 was previously executed (consequently the pixels determined to be 
background pixels at step 128 must be the identical pixels that were determined to be background pixels when step 
128 was previously executed). If the process is determined to have converged at step 132 for every block, then the 
current background and foreground colors are considered the extracted background and foreground colors for that 
block. If it is determined that the process has not converged for all blocks, step 1 28 is repeated for the blocks that have 
not converged until all blocks in the current grid have converged at step 132. 

At step 134, it is determined whether there are any smaller grids than the current grid. If there are, step 124 is 
repeated for the next smaller grid. If there are no smaller grids, at step 1 36 the background and foreground color images 
for the color document image are formed by combining the foreground and background colors of the blocks in the 
smallest grid. 

The process of Fig. 8 eliminates the problems discussed in conjunction with dividing the document into fixed block 
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sizes. For example, if the current block contains only background pixels, a small proportion of the foreground and 

a T B r!Z^ fT T^" 9 W ° Ck Wi " P ' 3y * Si 9 nificant role The -suLg'backgroun o£ wTbe the 

,! 0,0r0fth t ! P,Xelso,theblockTheresu "»n9 foreground colorwillbe the foregroundcoW of 
block. If however the current block contains pixels representing two nicely contrasted colors, the ^U^SS?S 
colo s identified for the larger block will have a negligible impact on the resulting clusters Proportion of 

* ro one ^ mbodlment ° f the Present invention, the steps of Fig. 8 are implemented on computer 10 Color imaqes 
are scanned by scanner 24 at twenty-four bits/pixel and three hundred pixels/inch. The sequence of grids of dec eTsino 
block sizes ,s butt by f,rst constructing the highest resolution grid using a block size of L.v "by ?Z?3STS! 
block size generates foreground and background colors at twenty-five pixels/inch. Successive grits Zl de TealTa 

ZTTJtT 5"^, mU ' tiPlyin9 b '° Ck WidUl hei9ht by ,OUr Until either the bl °<* width 9 o7heTght exceed hi 
th e 9 ,m?o h blocks located on the edge of the color image are moved inwards unti. they are entireXaiTwithin 
the image boundaries. The variable a, which is used in step 1 30, equals 0 9 

inuJZZT H be w ba <* ground and foreground images of a color document image are extracted by the present 
invention by dividing the document image into a plurality of multiscaled grids of blocks The cobrs of L ch blol o 
each gnd are determined by using biasing information from a corresponding block 

Several embodiments of the present invention are specifically illustrated and/or described herein However it will 
be apprecated that modifications and variations of the present invention are covered by the abc^^^S'iSiS 
the purview of the appended claims without departing from the spirit and intended scope of the invention 

Claims 

1 ' sCsT ° f 6XtraC,in9 8 ,0regr ° Und ima 9 6 and a background image from a color document image comprising the 

fiuramv of ^ t SL^nH 0 t < r! Jment i'? 96 T 3 P,UraNty °' multiscaled 9*., wherein each of the grids includes 
a plu alrty of b ocks and the resolution of the plurality of blocks increases for each successive grid and wherein 
the plurahty of gnds include a largest grid and a smallest grid- wnerein 

(b) determining a first background color and a first foreground color of each block of the largest grid- 

(c) determining a second background color and a second foreground color of each block of a next successive 
grid using biasing information from a corresponding block- successive 

(d) repeating steps (c) and (d) if the second background colors and the second foreground colors of each 
block of the smallest grid has not been determined; and 

(e) forming the foreground and background images from the determined second background color and second 
foreground color of each block of the smallest grid. second 

2. The method of claim 1 , wherein step (c) comprises the step of: 

(c-1) initializing the second background color and the second foreground color of each block of the next sue 
cessive gnd to a third background color and a third foreground color of the corresponding b^ck 

3. The method of claim 2, wherein step (c) further comprises the steps of: 

LCurS; aT" 9 ™ * *** ^ * ^ SUCC6SSiVe iS a <°^™ d P*" <" - 
(c-3) updating the second background color and the second foreground color of each block of the next suc- 
cess.ve grid us.ng a weighted average based on the corresponding block. 

The method of claim 3, wherein step (c) further comprises the steps of: 

(c-4) determining whether each block of the next successive grid has converged and 

™T a,in9 HK tePS (C " 2) ' (C3) ' (C_4) (C ' 5) if 31 Step (C " 4 > il is determined'that each block of the next 
successive grid has not converged. 

The method of claim 1 wherein the corresponding block of each block of the next successive grid is a first block 
of the prev,ous gnd, wherein the first block includes a center of the block of the next successive grid 

The method of claim 3, wherein step (c-3) comprises the following steps: 
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(c-3-1) determining an average foreground color of every pixel that is determined to be a foreground pixel in 
step (c-2); 

(c-3-2) forming a first operand by multiplying the average foreground color by a, wherein 0 < a < 1 ; 
(c-3-3) forming a second operand by multiplying the third foreground color by (1-a); and 
s (c-3-4) adding together the first operand and the second operand. 

7. The method of claim 1 , wherein step (e) comprises the steps of: 

(e-1) forming the foreground image by combining the determined second foreground colors of each block of 
io the smallest grid; and 

(e-2) forming the background image by combining the determined second background colors of each block 
of the smallest grid. 

8. The method of claim 3, wherein step (c-2) comprises the step of: 

15 

(c-2-1 ) comparing the distances of each pixel from the second background color and the second foreground 
color. 

9. A method for extracting foreground and background image information from a color document image comprising 
20 the steps of: 

(a) dividing the color document image into a plurality of grids, said plurality of grids varying from a first grid 
having a plurality of blocks of a largest size to a last grid having a plurality of blocks of a smallest size; 

(b) determining a first background color and a first foreground color of each block of the first grid; 

25 (c) determining a second background color and a second foreground color of each block of a next successive 

grid of the plurality of grids using biasing information from a corresponding block of a preceding grid; and 

(d) repeating steps (c) and (d) if the next successive grid is not the last grid. 

10. The method of claim 9, further comprising the step of: 

30 

(e) forming a foreground image and a background image from the determined second background colors and 
second foreground colors of each block of the last grid. 

11. The method of claim 10, wherein step (c) comprises the step of: 

35 

(c-1 ) initializing the second background color and the second foreground color of each block of the next suc- 
cessive grid to a third background color and a third foreground color of the corresponding block. 

12. The method of claim 11 , wherein step (c) further comprises the steps of: 

40 

(c-2) determining whether each pixel of each block of the next successive grid is a foreground pixel or a 
background pixel; and 

(c-3) updating the second background color and the second foreground color of each block of the next suc- 
cessive grid using a weighted average based on the corresponding block. 

45 

13. The method of claim 12, wherein step (c) further comprises the steps of: 

(c-4) determining whether each block of the next successive grid has converged; and 

(c-5) repeating steps (c-2), (c-3) t (c-4) and (c-5) if at step (c-4) it is determined that each block of the next 
50 successive grid has not converged. 

14. The method of claim 9, wherein the corresponding block ol each block of the next successive grid is a first block 
of an immediately preceding grid of the next successive grid, wherein the first block includes a center of the block 
of the next successive grid. 

55 

15. The method of claim 12, wherein step (c-3) comprises the following steps: 

(c-3-1 ) determining an average foreground color of every pixel that is determined to be a foreground pixel in 
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step (c-2); 

(c-3-2) forming a first operand by multiplying the average foreground color by «, wherein 0 < a < 1 

J?" 9 3 SeC ° nd °P erand b V multiplying the third foreground color by (1 -a)- and 
(c-3-4) adding together the first operand and the second operand. 

16. The method of claim 10, wherein step (e) comprises the steps of: 

Jhe'lLlTg^id^nd ,0re9r ° Und ' ma9e C ° mbinin9 det6rmined second Aground colors of each b.ock of 
S^toX" baCk9r ° md ima9e by C ° mbinin9 ,he de,erm ined second background colors of each block 

17. The method of claim 12, wherein step (c-2) comprises the step of; 

color ] C ° mparing the diStanC6S ° f each P jxel f rom the ^cond background color and the second foreground 

18. A system for extracting a foreground image and a background image from a color document image comprising: 

a storage device that stores the color document image; 
a central processing unit coupled to the storage device and programmed to 

aoTuramv ll^lT^ "T ' * **• "** ° f 9ridS from a first 9 rid h ^ing 

d^rrn nl! f t J I !f ^ * 3 laSt ^ haVi " 9 3 P 1 "^ of blocks <* * tallest size; 
de ermrne a f,rst background color and a first foreground color of each block of the first grid 

oft^u^ COl ° r 3 SSCOnd f ° regrOUnd CO, ° r ° f each block °f » "«i recessive grid 

19. The system of claim 18, wherein said processor is further programmed to: 

initialize the second background color and the second foreground color of each block of the next successive 
gr.d to a th.rd background color and a third foreground color of the corresponding block ^ess,ve 

20. The system of claim 19, wherein said processor is further programmed to: 

pS and" WhGther ^ PiX6i ° f ^ 6IOCk ° f n8Xt SUCC6SSiVe 9rid is 3 ,orearound P* el ^ a background 
update th< 

grid using a weighted average based on the corresponding block. 

21. The system of claim 18, wherein the corresponding block of each block of the next successive grid is a first block 
s^s^ 9 ° f SUCC8SSiVe 9rid ' Wh6rejn the flrSt bl ° Ck inC ' Udes a ^ « theb'ock o! the ne* 



30 



3S 



40 



iS 3 !! *AT_°_ nd . b f Ck9r ° Und COlOT andthe second foreground color of each block of the next successive 



22. 



irn^^^^ 



23. A system for extracting foreground and background image information from a color document image comprising: 
means for dividing the color document image into a plurality of grids, said plurality of grids varying from a first 

SJT^ltS ° f J't °l a ,ar9eSt S ' Ze ,0 3 ' aSt 9rid haVln9 3 pU " a,i * °< b '° cks of aTa l esTsle 
means or determining a f.rst background color and a first foreground color of each block of the first grid- and 

means for determining a second background color and a second foreground color o ea^h block ol a ne« 
success^ grid of the plurality of grids iterative.y using biasing information from a XreT^nT^Ta 
preceding gr.d unt.l the next successive grid is the last grid. corresponaing oiock of a 
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24. The system claim 23, further comprising: 

means for forming a foreground image and a background image from the determined second background 
colors and second foreground colors of each block of the last grid. 
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FIG. 1 
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FIG. 3 



50 



AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 



aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ^58 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa J 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 



r 



60 



V 



54 



j: 



52 



EZL 




r. 



a aa aa aa^aaa aa aa aa aa aa ajjaaa aa aa aa aa aa aaaaa aaaa aa _ 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa- 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 

TaTalftaaaa saaa aTaliaiftaa a3 SaaTTalTaliaaa aa ~ 

_ jflaaflflflaaaaaaaaaaafliaaflaaaaaaaaaaflaaaa_ 

r 'i aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 

" TinooaSi^ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-- 

/ f sf|iill\ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 



11 



BNSDOCID: <EP_ 



_0886242A1 J > 



EP 0 886 242 A1 



FIG. 4 



AAAAAAAAAAAAAAAAAAAAAAAAAA 



50 



aaaaaaaaaa aaaaaaaaaaaa aaaaaaaaaaaaaaaa**aa*aaa 
aaaaaaaaaaaaaaaaaaddddadadaaaaadd 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
addaaaaaaaaaaaaaaadaaadaaaadaaaaaaa 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aa ^3aaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaa 

ddddddddddddddddddaa ddddddddddadddaaaa 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aadaaaaaaaadaaaaaaaadaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 



EZL 



/555 a an an \ 



JZ3 



>AAAA 



Hit 



71 



■ ■ 



a aa aa a_a a aa aa aa aa aa a_a a_a a aa aa aa aa aa aa aa a a a a a a 3 a a 

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaalal" 
-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa- 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
^aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa- 
_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aa^^alala^a^aTaSaalalalsaaTaTaTaaaaa " 
— -a-aaflaflaaaaaaaaaaaajaaa3aaaflaaaaaaaajaa_ 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa- 
jiy J. ajiaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 



■70 



* EP 0 886 242 A1 

\ 

FIG. 5 













8l_ 


























82 




























IS 





















70 



FIG. 6 













91 
























92 






























93 





















81 



13 

BNSDOCID: <EP 0886242A1 _l_> 



EP 0 886 242 A1 



FIG. 7 



DIVIDE IMAGE INTO A PLURALITY 
OF MULTISCALED GRIDS 



-100 



i£l E n , ! 1I n N , E n BACKGn0UND AN D FOREGROUND 
COLORS OF EACH BLOCK OF LARGEST GRID 



-1 02 



mi noc nc c*™ n , D EE n S I . NE BACKGROUND AND FOREGROUND 




FORM FOREGROUND AND 
BACKGROUND IMAGES 



-10B 



EP 0 886 242 A1 



FIG. 8 



DIVIDE DOCUMENT INTO A PLURALITY 
OF MULTISCALED GRIDS 



-120 



DETERMINE FOREGROUND AND BACKGROUND 
COLOR OF EACH BLOCK IN LARGEST GRID 
USING BICOLOR PROCESS OF FIG. 2 



BEGIN DETERMINATION OF THE 
NEXT SMALLER GRID 



-122 



-124 



INITIALIZE BACKGROUND AND FOREGROUND 
COLORS TO COLORS OF CORRESPONDING 
BLOCK OF PREVIOUS GRID 



-126 





FOR EACH PIXEL. DETERMINE WHETHER 
PIXEL IS FOREGROUND OR 
BACKGROUND 


1 




UPDATE FOREGROUND 
AND BACKGROUND COLORS USING 
WEIGHTED AVERAGE 


NO / ^< 



-128 



-130 



YES, 



SMALLER 
GRIDS? 



NO 



FORM FOREGROUND AND 
BACKGROUND IMAGES 



-136 



15 



BNSDOCID: <EP 0886242A1_I_> 



EP0 886 242 A1 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Nurnbet 

EP 93 30 4673 



DOCUMENTS CONSIDERE D TO BE RELEVANT 



Category 



'P. A 



Citatior of document M\h indication, where appropriate 
of relevant passages 



US 5 640 468 A (HSU SHIN-YI ) 17 June 1907 

* abstract * 

* column 13, line 36 - line 63 * 

EP 0 724 229 A (CANON KK ) 31 July 1996 

* abstract * 

* column 4 y 1 i ne 35 - col umn 5 . line 10 * 

* column 14, line 46 - line 53 * 

crlrMTL^ 1 "MLTIRESOLUTION COLOR IMAGE 
SEGMENTATION 

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND 
MACHINE INTELLIGENCE , 
vol. 16, no. 7, 1 July 1994. pages 
689-700, XP000454190 

* abstract * 

* page 690, paragraph 'Using the 
Multiresolution Scheme to Refine the 
Segmentation' * 



The present search report has oeen drawn up tot all claims 



Pelevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION <lnt.Cl.6) 



1-24 



1-24 



1-24 



G06T5/00 



TECHNICAL FIELDS 
SEARCHED (lnt.CI.6) 



G06T 



Place of search 

THE HAGUE 



Date of corr^iencn of the search 

2 October 1998 



CATEGORY OF CITED DOCUMENTS 

x particu-ariy relevant If taken alone 

Y particular relevant it combined with another 

document of trie same category 
A technciogtca I background 
O : non^ritten disclosure 
P intermediate document 



Gonzalez Ordonez, 0 



T : theory or principle underlying the invention 
e • earlier patent document. &ut published on or 

after the liltng date 
D : document cited »n the application 
L : document cited tor otrw reasons 

S : member at the same psrenr family, correspondina 
document y 



16 

BNSDOCID: <EP 0886242A1J_> 



